Skip to content

INTPYTHON-355 Add transaction support #313

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 17 commits into
base: main
Choose a base branch
from

Conversation

aclark4life
Copy link
Collaborator

@aclark4life aclark4life commented Jun 4, 2025

I didn't try running the test suite on a sharded cluster. Maybe this could be done at a later date.

@aclark4life aclark4life changed the title Check for replica set or sharded cluster INTPYTHON-355 Check for replica set or sharded cluster Jun 5, 2025
@@ -97,6 +95,22 @@ class DatabaseFeatures(BaseDatabaseFeatures):
"expressions.tests.ExpressionOperatorTests.test_lefthand_transformed_field_bitwise_or",
}

@cached_property
def supports_transactions(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the purpose of this property? Is it just for the test suite or do apps rely on it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not just for testing.

In this case, rather than declare False or True for supports_transactions we dynamically set True or False based on what type of MongoDB we are connected to.

Then DatabaseFeatures is passed to DatabaseWrapper IIRC and the runtime behavior of Django changes accordingly.

Based on this: https://github.com/django/django/blob/main/django/db/backends/base/features.py#L419-L431

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It depends on what we want to implement. If we want to dynamically enable transactions based on whether or not the database supports it, we could use it internally. Otherwise, we'll have to introduce a separate setting the user sets to indicate whether or not they want to use transactions, and raise an error if it's not supported.

It's also used by Django in a couple places. For example, to determine whether or not to use transactions to speed up tests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is that there are certain storage engines where a replica set or sharded cluster does not actually support transactions so this check isn't 100% accurate. That said, those should be rare so I'm fine with the current "hello" based approach. We can always revisit later if it causes an issue.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in c8f8881 (assuming wired tiger storage engine is correct or we could also check to make sure it is not mmapv1)

@@ -97,6 +95,22 @@ class DatabaseFeatures(BaseDatabaseFeatures):
"expressions.tests.ExpressionOperatorTests.test_lefthand_transformed_field_bitwise_or",
}

@cached_property
def supports_transactions(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is that there are certain storage engines where a replica set or sharded cluster does not actually support transactions so this check isn't 100% accurate. That said, those should be rare so I'm fine with the current "hello" based approach. We can always revisit later if it causes an issue.

@aclark4life aclark4life changed the title INTPYTHON-355 Check for replica set or sharded cluster INTPYTHON-355 Add transaction support Jun 6, 2025
@aclark4life aclark4life marked this pull request as ready for review June 6, 2025 23:37
@aclark4life

This comment was marked as resolved.

@aclark4life aclark4life requested a review from timgraham June 6, 2025 23:39
@timgraham timgraham force-pushed the INTPYTHON-355 branch 2 times, most recently from e68f8c9 to 4a8209a Compare June 7, 2025 14:37
Comment on lines 21 to 58
name: Django Test Suite
runs-on: ubuntu-latest
steps:
- name: Checkout django-mongodb-backend
uses: actions/checkout@v4
with:
persist-credentials: false
- name: install django-mongodb-backend
run: |
pip3 install --upgrade pip
pip3 install -e .
- name: Checkout Django
uses: actions/checkout@v4
with:
repository: 'mongodb-forks/django'
ref: 'mongodb-5.2.x'
path: 'django_repo'
persist-credentials: false
- name: Install system packages for Django's Python test dependencies
run: |
sudo apt-get update
sudo apt-get install libmemcached-dev
- name: Install Django and its Python test dependencies
run: |
cd django_repo/tests/
pip3 install -e ..
pip3 install -r requirements/py3.txt
- name: Copy the test settings file
run: cp .github/workflows/mongodb_settings.py django_repo/tests/
- name: Copy the test runner file
run: cp .github/workflows/runtests.py django_repo/tests/runtests_.py
- name: Start MongoDB
uses: supercharge/[email protected]
with:
mongodb-version: 6.0
mongodb-replica-set: test-rs
- name: Run tests
run: python3 django_repo/tests/runtests.py --settings mongodb_settings -v 2

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium test

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
@timgraham timgraham force-pushed the INTPYTHON-355 branch 3 times, most recently from 8205175 to 1c784eb Compare June 8, 2025 02:08
@aclark4life aclark4life requested a review from Jibola June 9, 2025 17:44
@timgraham timgraham marked this pull request as draft June 9, 2025 19:28
@aclark4life
Copy link
Collaborator Author

investigate remaining test failures

Referring to tests defined in _django_test_expected_failures_transactions or something else ?

@timgraham timgraham force-pushed the INTPYTHON-355 branch 2 times, most recently from 763a039 to f007803 Compare June 10, 2025 15:22
@timgraham timgraham force-pushed the INTPYTHON-355 branch 2 times, most recently from d02ce65 to 4e48cec Compare June 13, 2025 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants